Comparison of scoring systems for invasive pests using ROC analysis and Monte Carlo simulation

Different international plant protection organisations advocate different schemes for conducting pest risk assessments. Most of these schemes use structured questionnaire in which experts are asked to score several items using an ordinal scale. The scores are then combined using a range of procedures, such as simple arithmetic mean, weighted averages, multiplication of scores, and cumulative sums. The most useful schemes will correctly identify harmful pests and identify ones that are not. As the quality of a pest risk assessment can depend on the characteristics of the scoring system used by the risk assessors (i.e., on the number of points of the scale and on the method used for combining the component scores), it is important to assess and compare the performance of different scoring systems. In this article, we proposed a new method for assessing scoring systems. Its principle is to simulate virtual data using a stochastic model and, then, to estimate sensitivity and specificity values from these data for different scoring systems. The interest of our approach was illustrated in a case study where several scoring systems were compared. Data for this analysis were generated using a probabilistic model describing the pest introduction process. The generated data were then used to simulate the outcome of scoring systems and to assess the accuracy of the decisions about positive and negative introduction. The results showed that ordinal scales with at most 5 or 6 points were sufficient and that the multiplication-based scoring systems performed better than their sum-based counterparts. The proposed method could be used in the future to assess a great diversity of scoring systems.

Prognostic factors for malignant mesothelioma in 142 patients : validation of CALGB and EORTC prognostic scoring systems

Background The incidence of malignant mesothelioma is increasing. There is the perception that survival is worse in the UK than in other countries. However, it is important to compare survival in different series based on accurate prognostic data. The European Organisation for Research and Treatment of Cancer (EORTC) and the Cancer and Leukaemia Group B (CALGB) have recently published prognostic scoring systems. We have assessed the prognostic variables, validated the EORTC and CALGB prognostic groups, and evaluated survival in a series of 142 patients. Methods Case notes of 142 consecutive patients presenting in Leicester since 1988 were reviewed. Univariate analysis of prognostic variables was performed using a Cox proportional hazards regression model. Statistically significant variables were analysed further in a forward, stepwise multivariate model. EORTC and CALGB prognostic groups were derived, Kaplan-Meier survival curves plotted, and survival rates were calculated from life tables. Results Significant poor prognostic factors in univariate analysis included male sex, older age, weight loss, chest pain, poor performance status, low haemoglobin, leukocytosis, thrombocytosis, and non-epithelial cell type (p<0.05). The prognostic significance of cell type, haemoglobin, white cell count, performance status, and sex were retained in the multivariate model. Overall median survival was 5.9 (range 0-34.3) months. One and two year survival rates were 21.3% (95% CI 13.9 to 28.7) and 3.5% (0 to 8.5), respectively. Median, one, and two year survival data within prognostic groups in Leicester were equivalent to the EORTC and CALGB series. Survival curves were successfully stratified by the prognostic groups. Conclusions This study validates the EORTC and CALGB prognostic scoring systems which should be used both in the assessment of survival data of series in different countries and in the stratification of patients into randomised clinical studies.

Nation & internation practices in decision support tools in road asset management

This document provides a review of international and national practices in investment decision support tools in road asset management. Efforts were concentrated on identifying analytic frameworks, evaluation methodologies and criteria adopted by current tools. Emphasis was also given to how current approaches support Triple Bottom Line decision-making. Benefit Cost Analysis and Multiple Criteria Analysis are principle methodologies in supporting decision-making in Road Asset Management. The complexity of the applications shows significant differences in international practices. There is continuing discussion amongst practitioners and researchers regarding to which one is more appropriate in supporting decision-making. It is suggested that the two approaches should be regarded as complementary instead of competitive means. Multiple Criteria Analysis may be particularly helpful in early stages of project development, say strategic planning. Benefit Cost Analysis is used most widely for project prioritisation and selecting the final project from amongst a set of alternatives. Benefit Cost Analysis approach is useful tool for investment decision-making from an economic perspective. An extension of the approach, which includes social and environmental externalities, is currently used in supporting Triple Bottom Line decision-making in the road sector. However, efforts should be given to several issues in the applications. First of all, there is a need to reach a degree of commonality on considering social and environmental externalities, which may be achieved by aggregating the best practices. At different decision-making level, the detail of consideration of the externalities should be different. It is intended to develop a generic framework to coordinate the range of existing practices. The standard framework will also be helpful in reducing double counting, which appears in some current practices. Cautions should also be given to the methods of determining the value of social and environmental externalities. A number of methods, such as market price, resource costs and Willingness to Pay, are found in the review. The use of unreasonable monetisation methods in some cases has discredited Benefit Cost Analysis in the eyes of decision makers and the public. Some social externalities, such as employment and regional economic impacts, are generally omitted in current practices. This is due to the lack of information and credible models. It may be appropriate to consider these externalities in qualitative forms in a Multiple Criteria Analysis. Consensus has been reached in considering noise and air pollution in international practices. However, Australia practices generally omitted these externalities. Equity is an important consideration in Road Asset Management. The considerations are either between regions, or social groups, such as income, age, gender, disable, etc. In current practice, there is not a well developed quantitative measure for equity issues. More research is needed to target this issue. Although Multiple Criteria Analysis has been used for decades, there is not a generally accepted framework in the choice of modelling methods and various externalities. The result is that different analysts are unlikely to reach consistent conclusions about a policy measure. In current practices, some favour using methods which are able to prioritise alternatives, such as Goal Programming, Goal Achievement Matrix, Analytic Hierarchy Process. The others just present various impacts to decision-makers to characterise the projects. Weighting and scoring system are critical in most Multiple Criteria Analysis. However, the processes of assessing weights and scores were criticised as highly arbitrary and subjective. It is essential that the process should be as transparent as possible. Obtaining weights and scores by consulting local communities is a common practice, but is likely to result in bias towards local interests. Interactive approach has the advantage in helping decision-makers elaborating their preferences. However, computation burden may result in lose of interests of decision-makers during the solution process of a large-scale problem, say a large state road network. Current practices tend to use cardinal or ordinal scales in measure in non-monetised externalities. Distorted valuations can occur where variables measured in physical units, are converted to scales. For example, decibels of noise converts to a scale of -4 to +4 with a linear transformation, the difference between 3 and 4 represents a far greater increase in discomfort to people than the increase from 0 to 1. It is suggested to assign different weights to individual score. Due to overlapped goals, the problem of double counting also appears in some of Multiple Criteria Analysis. The situation can be improved by carefully selecting and defining investment goals and criteria. Other issues, such as the treatment of time effect, incorporating risk and uncertainty, have been given scant attention in current practices. This report suggested establishing a common analytic framework to deal with these issues.

Constructing quality : the Hong Kong Housing Department journey

This paper explores the ‘journey’ along the ‘never ending quality road’ undertaken by the Hong Kong Housing Department over the last 15 years. It briefly covers the early history of public housing in Hong Kong, the catalytic effect brought about by the discovery of the infamous 26 sub-standard blocks in the mid-80s leading to the subsequent major improvements to process control and structural quality in the period 1985-1990. It then moves onto a discussion of initiatives taken since 1991, including the formation of the List of Building Contractors and the implementation of the Performance Assessment Scoring System (PASS). The paper ends with a discussion of the current status of quality issues within the Department and touches on future initiatives being developed to further enhance the quality of public housing in Hong Kong.

Development and validation of 3-Minute Nutrition Screening (3-MinNS) tool for acute hospital patients in Singapore

It is important to detect and treat malnutrition in hospital patients so as to improve clinical outcome and reduce hospital stay. The aim of this study was to develop and validate a nutrition screening tool with a simple and quick scoring system for acute hospital patients in Singapore. In this study, 818 newly admitted patients aged above 18 years old were screened using five parameters that contribute to the risk of malnutrition. A dietitian blinded to the nutrition screening score assessed the same patients using the reference standard, Subjective Global Assessment (SGA) within 48 hours. The sensitivity and specificity were established using the Receiver Operator Characteristics (ROC) curve and the best cutoff scores determined. The nutrition parameter with the largest Area Under the ROC Curve (AUC) was chosen as the final screening tool, which was named 3-Minute Nutrition Screening (3-MinNS). The combination of the parameters weight loss, intake and muscle wastage (3-MinNS), gave the largest AUC when compared with SGA. Using 3-MinNS, the best cutoff point to identify malnourished patients is three (sensitivity 86%, specificity 83%). The cutoff score to identify subjects at risk of severe malnutrition is five (sensitivity 93%, specificity 86%). 3-Minute Nutrition Screening is a valid, simple and rapid tool to identify patients at risk of malnutrition in Singapore acute hospital patients. It is able to differentiate patients at risk of moderate malnutrition and severe malnutrition for prioritization and management purposes.

Measuring quality : how does this improve construction performance?

The paper charts the history and development of the Hong Kong Housing Department (HKHD) Performance Assessment Scoring System (PASS) from 1990 to the present day and examines its effect on facilitating change to the quality of construction work of building contractors engaged in the production of public sector housing projects Hong Kong. The paper builds partly on empirical research carried out by the author as part of a doctoral thesis from 2000 to 2005, on experiential knowledge and also on some relevant case studies. The outcomes from this earlier research and validation of PASS based on results derived from the system since the research was originally undertaken are of benefit to practitioners and academics working and studying in the areas of performance assessment and organisational culture and change. The conclusions presented in the paper further underpin the connection established in previous research between strong organisational culture and project and corporate success. Organisational culture was measured using a survey instrument originally developed by Denison and Neale (1994), adapted for the environment of the study, and corporate success was measured by the PASS system mentioned above. The major results of the original study indicate that there is significant linkage between strong organisational cultures and business success and the detailed findings were that, (1) strong organisational culture was positively associated a high level of company effectiveness, (2) a high level of company effectiveness was positively associated with the cultural traits of ‘consistency’, ‘adaptability’ and ‘mission’, and (3) a high level of company effectiveness was positively associated with the combined cultural traits represented by the dimensions of ‘external focus’ and ‘stable culture’. Several opportunities to take forward this research have been identified, including extending the study to other countries and also longitudinally re-evaluating some of the original case studies to ascertain how organisational cultures have changed or further developed in relation to the changing construction climate in Hong Kong.

Assessing organizational culture? value management to the rescue

As part of a Doctor of Business Administration degree programme jointly run by Curtin University, Perth, Australia and Lingnan University, Hong Kong, a research thesis relating organizational effectiveness to the organizational culture of Hong Kong construction firms involved in public housing is being undertaken. Organizational effectiveness is measured by the Housing Department (HD) Performance Assessment Scoring System (PASS) and organizational culture traits and strengths have been measured by using the Denison Organizational Culture Survey (OCS), developed by Daniel Denison and William S. Neale and based on 16 years of research involving over 1,000 organizations. The PASS scores of building contractors are compared with the OCS scores to determine if there is any significant correlation between highly effective companies and particular organizational strengths and traits. Profiles are then drawn using the Denison Model and can be compared against ‘norms’ for the industry sector on which the survey has been carried out. The next stage of the work is to present the results of the survey to individual companies, conduct focus group interviews to test the results, discover more detail on that company’s culture and discuss possible actions based on the results. It is in this latter stage that certain value management techniques may well prove very useful.

Construction performance and organisational culture

The purpose of this paper is to summarize the outcomes of a detailed research study carried out as part of the fulfilment of a doctoral programme which examined the relationships between, and impacts of organisational culture on construction performance within a Hong Kong context. The research used a mixed methodology approach consisting of an organisational culture survey using an adapted validated and reliable measurement instrument (the Denison Organisational Culture Survey), mini-case studies in four Hong Kong construction companies and correlated the derived culture scores against performance scores measured by the Hong Kong Housing Department Performance Assessment Scoring System (PASS). The significance of the research was to advance knowledge of the importance of organisational culture strength as a performance driver in the construction industry and the further proof of the culture performance links using a set of measures of the latter which were not financially-based. The findings of the research make a contribution to theory by further validating the work by Denison (1990) and others, not only in that a successful link between organisational culture and performance was demonstrated, but it also identifies particular cultural factors in organisations that appear to be significantly responsible for achieving successful outcomes and reveals opportunities for further research into the organisational culture of construction companies Keywords: organisational culture, construction performance, business success.

The Validity of the Brief Michigan Alcohol Screening Test (bMAST) as a Problem Drinking Severity Measure

Objective: The Brief Michigan Alcoholism Screening Test (bMAST) is a 10-item test derived from the 25-item Michigan Alcoholism Screening Test (MAST). It is widely used in the assessment of alcohol dependence. In the absence of previous validation studies, the principal aim of this study was to assess the validity and reliability of the bMAST as a measure of the severity of problem drinking. Method: There were 6,594 patients (4,854 men, 1,740 women) who had been referred for alcohol-use disorders to a hospital alcohol and drug service who voluntarily participated in this study. Results: An exploratory factor analysis defined a two-factor solution, consisting of Perception of Current Drinking and Drinking Consequences factors. Structural equation modeling confirmed that the fit of a nine-item, two-factor model was superior to the original one-factor model. Concurrent validity was assessed through simultaneous administration of the Alcohol Use Disorders Identification Test (AUDIT) and associations with alcohol consumption and clinically assessed features of alcohol dependence. The two-factor bMAST model showed moderate correlations with the AUDIT. The two-factor bMAST and AUDIT were similarly associated with quantity of alcohol consumption and clinically assessed dependence severity features. No differences were observed between the existing weighted scoring system and the proposed simple scoring system. Conclusions: In this study, both the existing bMAST total score and the two-factor model identified were as effective as the AUDIT in assessing problem drinking severity. There are additional advantages of employing the two-factor bMAST in the assessment and treatment planning of patients seeking treatment for alcohol-use disorders. (J. Stud. Alcohol Drugs 68: 771-779,2007)

Useful field of view predicts driving in the presence of distracters

Purpose. The Useful Field of View (UFOV(R)) test has been shown to be highly effective in predicting crash risk among older adults. An important question which we examined in this study is whether this association is due to the ability of the UFOV to predict difficulties in attention-demanding driving situations that involve either visual or auditory distracters. Methods. Participants included 92 community-living adults (mean age 73.6 +/- 5.4 years; range 65-88 years) who completed all three subtests of the UFOV involving assessment of visual processing speed (subtest 1), divided attention (subtest 2), and selective attention (subtest 3); driving safety risk was also classified using the UFOV scoring system. Driving performance was assessed separately on a closed-road circuit while driving under three conditions: no distracters, visual distracters, and auditory distracters. Driving outcome measures included road sign recognition, hazard detection, gap perception, time to complete the course, and performance on the distracter tasks. Results. Those rated as safe on the UFOV (safety rating categories 1 and 2), as well as those responding faster than the recommended cut-off on the selective attention subtest (350 msec), performed significantly better in terms of overall driving performance and also experienced less interference from distracters. Of the three UFOV subtests, the selective attention subtest best predicted overall driving performance in the presence of distracters. Conclusions. Older adults who were rated as higher risk on the UFOV, particularly on the selective attention subtest, demonstrated poorest driving performance in the presence of distracters. This finding suggests that the selective attention subtest of the UFOV may be differentially more effective in predicting driving difficulties in situations of divided attention which are commonly associated with crashes.

Non-destructive evaluation of articular cartilage defects using near-infrared (NIR) spectroscopy in osteoarthritic rat models and its direct relation to Mankin Score

Objective The aim of this study was to demonstrate the potential of near-infrared (NIR) spectroscopy for categorizing cartilage degeneration induced in animal models. Method Three models of osteoarthritic degeneration were induced in laboratory rats via one of the following methods: (i) menisectomy (MSX); (ii) anterior cruciate ligament transaction (ACLT); and (iii) intra-articular injection of mono-ido-acetete (1 mg) (MIA), in the right knee joint, with 12 rats per model group. After 8 weeks, the animals were sacrificed and tibial knee joints were collected. A custom-made nearinfrared (NIR) probe of diameter 5 mm was placed on the cartilage surface and spectral data were acquired from each specimen in the wavenumber range 4 000 – 12 500 cm−1. Following spectral data acquisition, the specimens were fixed and Safranin–O staining was performed to assess disease severity based on the Mankin scoring system. Using multivariate statistical analysis based on principal component analysis and partial least squares regression, the spectral data were then related to the Mankinscores of the samples tested. Results Mild to severe degenerative cartilage changes were observed in the subject animals. The ACLT models showed mild cartilage degeneration, MSX models moderate, and MIA severe cartilage degenerative changes both morphologically and histologically. Our result demonstrate that NIR spectroscopic information is capable of separating the cartilage samples into different groups relative to the severity of degeneration, with NIR correlating significantly with their Mankinscore (R2 = 88.85%). Conclusion We conclude that NIR is a viable tool for evaluating articularcartilage health and physical properties such as change in thickness with degeneration.

Characteristics of on-road driving performance of persons with central vision loss who use bioptic telescopes

Purpose. To compare the on-road driving performance of visually impaired drivers using bioptic telescopes with age-matched controls. Methods. Participants included 23 persons (mean age = 33 ± 12 years) with visual acuity of 20/63 to 20/200 who were legally licensed to drive through a state bioptic driving program, and 23 visually normal age-matched controls (mean age = 33 ± 12 years). On-road driving was assessed in an instrumented dual-brake vehicle along 14.6 miles of city, suburban, and controlled-access highways. Two backseat evaluators independently rated driving performance using a standardized scoring system. Vehicle control was assessed through vehicle instrumentation and video recordings used to evaluate head movements, lane-keeping, pedestrian detection, and frequency of bioptic telescope use. Results. Ninety-six percent (22/23) of bioptic drivers and 100% (23/23) of controls were rated as safe to drive by the evaluators. There were no group differences for pedestrian detection, or ratings for scanning, speed, gap judgments, braking, indicator use, or obeying signs/signals. Bioptic drivers received worse ratings than controls for lane position and steering steadiness and had lower rates of correct sign and traffic signal recognition. Bioptic drivers made significantly more right head movements, drove more often over the right-hand lane marking, and exhibited more sudden braking than controls. Conclusions. Drivers with central vision loss who are licensed to drive through a bioptic driving program can display proficient on-road driving skills. This raises questions regarding the validity of denying such drivers a license without the opportunity to train with a bioptic telescope and undergo on-road evaluation.

A phase III trial of docetaxel/carboplatin versus mitomycin C/ifosfamide/ cisplatin (MIC) or mitomycin C/vinblastine/cisplatin (MVP) in patients with advanced non-small-cell lung cancer : a randomised multicentre trial of the British Thoracic Oncology Group (BTOG1)

Background: Phase III studies suggest that non-small-cell lung cancer (NSCLC) patients treated with cisplatin-docetaxel may have higher response rates and better survival compared with other platinum-based regimens. We report the final results of a randomised phase III study of docetaxel and carboplatin versus MIC or MVP in patients with advanced NSCLC. Patients and methods: Patients with biopsy proven stage III-IV NSCLC not suitable for curative surgery or radiotherapy were randomised to receive four cycles of either DCb (docetaxel 75 mg/m 2, carboplatin AUC 6), or MIC/MVP (mitomycin 6 mg/m 2, ifosfamide 3 g/m 2 and cisplatin 50 mg/m 2 or mitomycin 6 mg/ m 2, vinblastine 6 mg/m 2 and cisplatin 50 mg/m 2, respectively), 3 weekly. The primary end point was survival, secondary end points included response rates, toxicity and quality of life. Results: The median follow-up was 17.4 months. Overall response rate was 32% for both arms (partial response = 31%, complete response = 1%); 32% of MIC/MVP and 26% of DCb patients had stable disease. One-year survival was 39% and 35% for DCb and MIC/MVP, respectively. Two-year survival was 13% with both arms. Grade 3/4 neutropenia (74% versus 43%, P < 0.005), infection (18% versus 9%, P = 0.01) and mucositis (5% versus 1%, P = 0.02) were more common with DCb than MIC/MVP. The MIC/MVP arm had significant worsening in overall EORTC score and global health status whereas the DCb arm showed no significant change. Conclusions: The combination of DCb had similar efficacy to MIC/MVP but quality of life was better maintained. © 2006 European Society for Medical Oncology.

Angiogenesis is an independent prognostic factor in malignant mesothelioma

Angiogenesis is essential for tumour growth beyond 1 to 2 mm in diameter. The clinical relevance of angiogenesis, as assessed by microvessel density (MVD), is unclear in malignant mesothelioma (MM). Immunohistochemistry was performed on 104 archival, paraffin-embedded, surgically resected MM samples with an anti-CD34 monoclonal antibody, using the Streptavidin-biotin complex immunoperoxidase technique. 93 cases were suitable for microvessel quantification. MVD was obtained from 3 intratumoural hotspots, using a Chalkley eyepiece graticule at × 250 power. MVD was correlated with survival by Kaplan-Meier and log-rank analysis. A stepwise, multivariate Cox model was used to compare MVD with known prognostic factors and the EORTC and CALGB prognostic scoring systems. Overall median survival from the date of diagnosis was 5.0 months. Increasing MVD was a poor prognostic factor in univariate analysis (P = 0.02). Independent indicators of poor prognosis in multivariate analysis were non-epithelial cell type (P = 0.002), performance status > 0 (P = 0.003) and increasing MVD (P = 0.01). In multivariate Cox analysis, MVD contributed independently to the EORTC (P = 0.006), but not to the CALGB (P = 0.1), prognostic groups. Angiogenesis, as assessed by MVD, is a poor prognostic factor in MM, independent of other clinicopathological variables and the EORTC prognostic scoring system. Further work is required to assess the prognostic importance of angiogenic regulatory factors in this disease. © 2001 Cancer Research Campaign.

Tumor necrosis correlates with angiogenesis and is a predictor of poor prognosis in malignant mesothelioma

Objectives: Malignant mesothelioma (MM) is a fatal tumor of increasing incidence related to asbestos exposure. Microscopic tumor necrosis (TN) is a poor prognostic factor in solid tumors, but it has not been characterized in MM. We wished to evaluate the incidence of TN in MM and its correlations with clinicopathologic factors, angiogenesis, and survival. Methods: TN was graded in 171 routine formalin-fixed, paraffin-embedded hematoxylin-eosinstained tumor sections by two independent observers. Angiogenesis was assessed by the microvessel count (MVC) of CD34 immunostained sections. TN was correlated with survival by Kaplan-Meier and log-rank analysis, and stepwise, multivariate Cox models were used to compare TN with angiogenesis and established prognostic factors and prognostic scoring systems. Results: TN was identified in 39 cases (22.8%) and correlated with low hemoglobin (p = 0.01), thrombocytosis (p = 0.04), and high MVC (p = 0.02). TN was a poor prognostic factor in univariate analysis (p = 0.008). Patients with TN had a median survival of 5.3 months vs 8.3 months in negative cases. Independent indicators of poor prognosis in multivariate analysis were nonepithelioid cell type (p = 0.0001), performance status > 0 (p = 0.007), and increasing MVC (p = 0.004) but not TN. TN contributed independently to the European Organisation for Research and Treatment of Cancer (EORTC) [p = 0.03] and to the Cancer and Leukemia Group B (CALGB) [p = 0.03] prognostic groups in respective multivariate Cox analyses. Conclusions: TN correlates with angiogenesis and is a poor prognostic factor in MM. TN contributes to the EORTC and CALGB prognostic scoring systems.